Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(monitor): add reward service error metric #2905

Merged
merged 4 commits into from
Dec 20, 2023
Merged

feat(monitor): add reward service error metric #2905

merged 4 commits into from
Dec 20, 2023

Conversation

rickstaa
Copy link
Contributor

@rickstaa rickstaa commented Nov 10, 2023 •
edited
Loading

What does this pull request do? Could you explain your changes? (required)

This pull request introduces the reward_call_errors Prometheus metric to enhance user monitoring capabilities. The metric is added to the localhost:7935/metrics endpoint.

Specific updates (required)

  • Added a new metric, reward_call_error, in census.go.
  • Implemented logic in rewardservice.go to set the reward_call_error metric when monitoring is enabled and a reward call fails.

How did you test each of these updates (required)

I haven't had the chance to run tests on the feature yet. I'd appreciate some guidance on the optimal testing approach for this new functionality.

Does this pull request close any open issues?

No, it does not.

Checklist:

  • Read the contribution guide
  • make runs successfully
  • All tests in ./test.sh pass
  • README and other documentation updated - Documentation for the new metrics is pending and will be added.
  • Pending changelog updated

Test Logs

Test Logs 
++ go list ./...
++ grep -v test/e2e
+ go test -coverprofile cover.out github.com/livepeer/go-livepeer/build github.com/livepeer/go-livepeer/clog github.com/livepeer/go-livepeer/cmd/devtool github.com/livepeer/go-livepeer/cmd/devtool/devtool github.com/livepeer/go-livepeer/cmd/livepeer github.com/livepeer/go-livepeer/cmd/livepeer/starter github.com/livepeer/go-livepeer/cmd/livepeer_bench github.com/livepeer/go-livepeer/cmd/livepeer_cli github.com/livepeer/go-livepeer/cmd/livepeer_router github.com/livepeer/go-livepeer/cmd/scripts github.com/livepeer/go-livepeer/cmd/simple_auth_server github.com/livepeer/go-livepeer/common github.com/livepeer/go-livepeer/core github.com/livepeer/go-livepeer/crypto github.com/livepeer/go-livepeer/discovery github.com/livepeer/go-livepeer/eth github.com/livepeer/go-livepeer/eth/blockwatch github.com/livepeer/go-livepeer/eth/contracts github.com/livepeer/go-livepeer/eth/types github.com/livepeer/go-livepeer/eth/watchers github.com/livepeer/go-livepeer/monitor github.com/livepeer/go-livepeer/net github.com/livepeer/go-livepeer/pm github.com/livepeer/go-livepeer/server github.com/livepeer/go-livepeer/verification
?   	github.com/livepeer/go-livepeer/build	[no test files]
# github.com/livepeer/lpms/ffmpeg
extras.c: In function ‘lpms_compare_sign_bypath’:
extras.c:202:13: warning: implicit declaration of function ‘avfilter_compare_sign_bypath’; did you mean ‘lpms_compare_sign_bypath’? [-Wimplicit-function-declaration]
  202 |   int ret = avfilter_compare_sign_bypath(signpath1, signpath2);
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |             lpms_compare_sign_bypath
extras.c: In function ‘lpms_compare_sign_bybuffer’:
extras.c:213:13: warning: implicit declaration of function ‘avfilter_compare_sign_bybuff’; did you mean ‘lpms_compare_sign_bybuffer’? [-Wimplicit-function-declaration]
  213 |   int ret = avfilter_compare_sign_bybuff(buffer1, len1, buffer2, len2);
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |             lpms_compare_sign_bybuffer
?   	github.com/livepeer/go-livepeer/cmd/devtool	[no test files]
?   	github.com/livepeer/go-livepeer/cmd/devtool/devtool	[no test files]
ok  	github.com/livepeer/go-livepeer/clog	0.008s	coverage: 69.5% of statements
# github.com/livepeer/go-livepeer/eth/watchers.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-3184988398/000021.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-3184988398/000021.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

# github.com/livepeer/go-livepeer/eth.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-1388600829/000025.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-1388600829/000025.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

# github.com/livepeer/go-livepeer/common.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-4231523545/000018.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-4231523545/000018.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

# github.com/livepeer/go-livepeer/cmd/livepeer_cli.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-1643420494/000012.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-1643420494/000012.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

# github.com/livepeer/go-livepeer/cmd/livepeer/starter.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-3205396144/000017.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-3205396144/000017.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

# github.com/livepeer/go-livepeer/discovery.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-622095218/000018.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-622095218/000018.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

# github.com/livepeer/go-livepeer/cmd/livepeer.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-3710243153/000017.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-3710243153/000017.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

FAIL	github.com/livepeer/go-livepeer/cmd/livepeer [build failed]
FAIL	github.com/livepeer/go-livepeer/cmd/livepeer/starter [build failed]
?   	github.com/livepeer/go-livepeer/cmd/livepeer_bench	[no test files]
?   	github.com/livepeer/go-livepeer/cmd/livepeer_router	[no test files]
?   	github.com/livepeer/go-livepeer/cmd/scripts	[no test files]
?   	github.com/livepeer/go-livepeer/cmd/simple_auth_server	[no test files]
FAIL	github.com/livepeer/go-livepeer/cmd/livepeer_cli [build failed]
FAIL	github.com/livepeer/go-livepeer/common [build failed]
# github.com/livepeer/go-livepeer/verification.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-3628309214/000006.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-3628309214/000006.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

# github.com/livepeer/go-livepeer/core.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-3164035277/000006.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-3164035277/000006.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

FAIL	github.com/livepeer/go-livepeer/core [build failed]
?   	github.com/livepeer/go-livepeer/eth/contracts	[no test files]
?   	github.com/livepeer/go-livepeer/net	[no test files]
# github.com/livepeer/go-livepeer/server.test
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-1786446998/000006.o: in function `lpms_compare_sign_bypath':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:202: undefined reference to `avfilter_compare_sign_bypath'
/usr/bin/ld: /tmp/go-link-1786446998/000006.o: in function `lpms_compare_sign_bybuffer':
/home/ricks/go/pkg/mod/github.com/livepeer/[email protected]/ffmpeg/extras.c:213: undefined reference to `avfilter_compare_sign_bybuff'
collect2: error: ld returned 1 exit status

ok  	github.com/livepeer/go-livepeer/crypto	0.013s	coverage: 90.0% of statements
FAIL	github.com/livepeer/go-livepeer/discovery [build failed]
FAIL	github.com/livepeer/go-livepeer/eth [build failed]
ok  	github.com/livepeer/go-livepeer/eth/blockwatch	0.232s	coverage: 66.0% of statements
ok  	github.com/livepeer/go-livepeer/eth/types	0.008s	coverage: 66.2% of statements
FAIL	github.com/livepeer/go-livepeer/eth/watchers [build failed]
ok  	github.com/livepeer/go-livepeer/monitor	0.032s	coverage: 54.5% of statements
ok  	github.com/livepeer/go-livepeer/pm	4.878s	coverage: 83.5% of statements
FAIL	github.com/livepeer/go-livepeer/server [build failed]
FAIL	github.com/livepeer/go-livepeer/verification [build failed]
FAIL

@rickstaa rickstaa marked this pull request as draft November 10, 2023 14:00
@rickstaa rickstaa mentioned this pull request Nov 10, 2023
Open
@rickstaa
Copy link
Contributor Author

I'm currently stuck on testing my changes because of a few missing dependencies or a bit of a tangle with dependency versions (see test logs above). Any friendly guidance you could offer would be a huge help!

@rickstaa rickstaa marked this pull request as ready for review November 11, 2023 13:04
@codecov Codecov
Copy link

codecov bot commented Nov 11, 2023 •
edited
Loading

Codecov Report

Attention: 10 lines in your changes are missing coverage. Please review.

Comparison is base (8a8b0e3) 56.33991% compared to head (a4a4889) 56.33564%.

❗ Current head a4a4889 differs from pull request most recent head 8acadbc. Consider uploading reports for the commit 8acadbc to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@                 Coverage Diff                 @@
##              master       #2905         +/-   ##
===================================================
- Coverage   56.33991%   56.33564%   -0.00427%     
===================================================
  Files             89          89                 
  Lines          19464       19485         +21     
===================================================
+ Hits           10966       10977         +11     
- Misses          7889        7898          +9     
- Partials         609         610          +1
Files Coverage Δ
eth/rewardservice.go 72.46377% <25.00000%> (-3.65563%) ⬇️
monitor/census.go 63.33333% <63.15789%> (-0.00263%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a8b0e3...8acadbc. Read the comment docs.

Files Coverage Δ
eth/rewardservice.go 72.46377% <25.00000%> (-3.65563%) ⬇️
monitor/census.go 63.33333% <63.15789%> (-0.00263%) ⬇️

@eliteprox
Copy link
Collaborator

This change looks very helpful. I like the idea of getting metrics for reward call errors. I think it would be useful to have metrics for all blockwatcher errors too. I can help you with the testing/dependency issues if you need.

I've kept notes on debugging and running automated tests with VS Code: https://github.com/eliteprox/livepeer-notes/blob/main/testing%20notes.md

You'll need to run install_ffmpeg.sh first before running the tests. Here's some more detail on it and a sample launch.json for debugging the latest version of go-livepeer which has CUDA dependencies https://github.com/eliteprox/livepeer-notes/blob/main/README.md.

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 1, 2023

This change looks very helpful. I like the idea of getting metrics for reward call errors. I think it would be useful to have metrics for all blockwatcher errors too. I can help you with the testing/dependency issues if you need.

I've kept notes on debugging and running automated tests with VS Code: https://github.com/eliteprox/livepeer-notes/blob/main/testing%20notes.md

You'll need to run install_ffmpeg.sh first before running the tests. Here's some more detail on it and a sample launch.json for debugging the latest version of go-livepeer which has CUDA dependencies https://github.com/eliteprox/livepeer-notes/blob/main/README.md.

@eliteprox thanks a lot for your support ❤️. This pull request is the first of several I want to create to improve the reward calling mechanism. As I'm new to the project, could you maybe point me to the other blockwatcher errors you would like to have metrics for? I'm happy to tackle them in other pull requests.

About the tests, they only seem to fail on Ubuntu 20.04 but pass on Ubuntu 22.04 (See https://discord.com/channels/423160867534929930/1102991036080472247). I, therefore, suspect library incompatibilities, dependency conflicts or missing environmental variables cause them. The documents you gave will be beneficial in trying to find out how these errors can be prevented on older Ubuntu versions. I will first focus on testing this pull request on Ubuntu 22.04 so that it can be merged, but then I will check if I can fix the errors on Ubuntu 20.04 so that, in the future, we know what was going wrong.

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 1, 2023

Hey @eliteprox, maybe we can consider adding a brief VSCode debugging section or a link to a separate vscode debugging readme in CONTRIBUTING.md 🤔. Your documents could be pretty handy for that.

@eliteprox
Copy link
Collaborator

eliteprox commented Dec 5, 2023 •
edited
Loading

Hey @eliteprox, maybe we can consider adding a brief VSCode debugging section or a link to a separate vscode debugging readme in CONTRIBUTING.md 🤔. Your documents could be pretty handy for that.

I think the official docs for building from source are located here, but they are a bit hard to find and don't mention development tooling like VS Code. I think that will be the best place for it, but we should also link to it from CONTRIBUTING.md. This has been on my todo list for a while, but my github notes are just a first attempt at organizing the knowledge into a draft.

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 5, 2023

Hey @eliteprox, maybe we can consider adding a brief VSCode debugging section or a link to a separate vscode debugging readme in CONTRIBUTING.md 🤔. Your documents could be pretty handy for that.

I think the official docs for building from source are located here, but they are a bit hard to find and don't mention development tooling like VS Code. I think that will be the best place for it, but we should also link to it from CONTRIBUTING.md. This has been on my todo list for a while, but my github notes are just a first attempt at organizing the knowledge into a draft.

I agree that a VScode debug section in the official docs would also work 👍🏻. Thanks again for your help.

@leszko
Copy link
Contributor

leszko commented Dec 12, 2023

@rickstaa Thanks for the PR. I find it useful 🎖️ And thanks @eliteprox for the help here.

@rickstaa is this PR ready for review and merge? If yes, then please resolve the conflicts and then I'll review and merge it.

@leszko leszko self-requested a review December 12, 2023 14:28
@rickstaa rickstaa force-pushed the add_reward_service_error_monitor_parameter branch from 70fef5f to 1c8f6f5 Compare December 14, 2023 15:59
@rickstaa
Copy link
Contributor Author

@leszko, I just rebased this pull request on the main branch. However, I have yet to thoroughly test this pull request to see if the metric is correctly published. It should work looking at the code, but I would like to double-check before merging it 🤔.

@leszko
Copy link
Contributor

leszko commented Dec 15, 2023

@leszko, I just rebased this pull request on the main branch. However, I have yet to thoroughly test this pull request to see if the metric is correctly published. It should work looking at the code, but I would like to double-check before merging it 🤔.

Perfect. Please double-check if it works and then tag me for reviewing and merging. Thanks.

@rickstaa
Copy link
Contributor Author

Hello @eliteprox and @leszko,

Before testing if the metric shows up correctly, I wanted to ensure the tests succeeded and address any potential issues. However, when I attempted to run the tests on my Ubuntu 22.04 Laptop, I encountered a failure accompanied by the 'Remote transcoder took too long' error message.

I suspect this issue may be related to my laptop's lack of a dedicated GPU 🤔. Before proceeding, I want to confirm this hypothesis to avoid complications on my production PC.

I have attached both my system information and the test logs. Any insights or suggestions would be greatly appreciated.

Thank you so much for your help.

System information

  • OS: Ubuntu 22.04
  • Kernel: 6.2.0-39-generic
  • GPU: NVIDIA RTX A1000 Laptop GPU
  • CUDA toolkit version: 11.7

Test logs

Test logs ```bash [19:24:39] ricks@LAPTOP-UBUNTU22:~/development/personal/go-livepeer$ sh test.sh + go list ./... + grep -v test/e2e go: downloading github.com/testcontainers/testcontainers-go v0.26.0 go: downloading github.com/dop251/goja v0.0.0-20230806174421-c933cf95e127 go: downloading github.com/mattn/go-colorable v0.1.13 go: downloading github.com/status-im/keycard-go v0.2.0 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-isatty v0.0.17 go: downloading github.com/google/pprof v0.0.0-20230207041349-798e818bf904 go: downloading github.com/dlclark/regexp2 v1.7.0 go: downloading dario.cat/mergo v1.0.0 go: downloading github.com/cenkalti/backoff/v4 v4.2.1 go: downloading github.com/cpuguy83/dockercfg v0.3.1 go: downloading github.com/moby/patternmatcher v0.6.0 go: downloading github.com/moby/term v0.5.0 go: downloading github.com/docker/docker v24.0.6+incompatible go: downloading github.com/containerd/containerd v1.7.7 go: downloading github.com/opencontainers/image-spec v1.1.0-rc5 go: downloading github.com/magiconair/properties v1.8.7 go: downloading github.com/shirou/gopsutil/v3 v3.23.9 go: downloading github.com/containerd/log v0.1.0 go: downloading github.com/sirupsen/logrus v1.9.3 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/docker/distribution v2.8.2+incompatible go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/opencontainers/runc v1.1.5 go: downloading github.com/moby/sys/sequential v0.5.0 + go test -coverprofile cover.out github.com/livepeer/go-livepeer/build github.com/livepeer/go-livepeer/clog github.com/livepeer/go-livepeer/cmd/devtool github.com/livepeer/go-livepeer/cmd/devtool/devtool github.com/livepeer/go-livepeer/cmd/livepeer github.com/livepeer/go-livepeer/cmd/livepeer/starter github.com/livepeer/go-livepeer/cmd/livepeer_bench github.com/livepeer/go-livepeer/cmd/livepeer_cli github.com/livepeer/go-livepeer/cmd/livepeer_router github.com/livepeer/go-livepeer/cmd/scripts github.com/livepeer/go-livepeer/cmd/simple_auth_server github.com/livepeer/go-livepeer/common github.com/livepeer/go-livepeer/core github.com/livepeer/go-livepeer/crypto github.com/livepeer/go-livepeer/discovery github.com/livepeer/go-livepeer/eth github.com/livepeer/go-livepeer/eth/blockwatch github.com/livepeer/go-livepeer/eth/contracts github.com/livepeer/go-livepeer/eth/types github.com/livepeer/go-livepeer/eth/watchers github.com/livepeer/go-livepeer/monitor github.com/livepeer/go-livepeer/net github.com/livepeer/go-livepeer/pm github.com/livepeer/go-livepeer/server github.com/livepeer/go-livepeer/verification go: downloading pgregory.net/rapid v1.1.0 ? github.com/livepeer/go-livepeer/build [no test files] ok github.com/livepeer/go-livepeer/clog 0.019s coverage: 69.5% of statements ? github.com/livepeer/go-livepeer/cmd/devtool [no test files] ? github.com/livepeer/go-livepeer/cmd/devtool/devtool [no test files] ok github.com/livepeer/go-livepeer/cmd/livepeer 0.052s coverage: 55.5% of statements ? github.com/livepeer/go-livepeer/cmd/livepeer_bench [no test files] ? github.com/livepeer/go-livepeer/cmd/livepeer_router [no test files] ? github.com/livepeer/go-livepeer/cmd/scripts [no test files] ? github.com/livepeer/go-livepeer/cmd/simple_auth_server [no test files] ? github.com/livepeer/go-livepeer/eth/contracts [no test files] ? github.com/livepeer/go-livepeer/net [no test files] ok github.com/livepeer/go-livepeer/cmd/livepeer/starter 0.102s coverage: 5.4% of statements ok github.com/livepeer/go-livepeer/cmd/livepeer_cli 0.121s coverage: 0.6% of statements ok github.com/livepeer/go-livepeer/common 0.202s coverage: 66.6% of statements WARNING: decoder.c:290] Nvidia decoder not found; defaulting to software No such filter: 'scale_npp' ERROR: filter.c:107] Unable to parse video filters desc : Invalid argument ERROR: encoder.c:231] Unable to open video filter : Invalid argument ERROR: transcoder.c:238] Unable to open output : Invalid argument E1215 19:25:42.556881 37054 ffmpeg.go:1012] Transcoder Return : Invalid argument WARNING: decoder.c:290] Nvidia decoder not found; defaulting to software No such filter: 'scale_npp' ERROR: filter.c:107] Unable to parse video filters desc : Invalid argument ERROR: encoder.c:231] Unable to open video filter : Invalid argument ERROR: transcoder.c:238] Unable to open output : Invalid argument E1215 19:25:42.599519 37054 ffmpeg.go:1012] Transcoder Return : Invalid argument [libx264 @ 0x7fe1d468e100] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1d468e100] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1d4cb0b40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1d4cb0b40] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1d4bee500] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1d4bee500] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1d40e32c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1d40e32c0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1d468e100] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1d468e100] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1d468e100] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1d468e100] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1d468e100] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1d468e100] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1d468e100] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1d468e100] 8x8 transform intra:0.0% [libx264 @ 0x7fe1d468e100] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1d468e100] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1d468e100] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1d468e100] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1d468e100] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1d468e100] kb/s:36.44 [libx264 @ 0x7fe1d4cb0b40] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1d4cb0b40] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1d4cb0b40] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1d4cb0b40] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1d4cb0b40] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1d4cb0b40] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1d4cb0b40] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1d4cb0b40] 8x8 transform intra:0.0% [libx264 @ 0x7fe1d4cb0b40] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1d4cb0b40] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1d4cb0b40] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1d4cb0b40] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1d4cb0b40] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1d4cb0b40] kb/s:36.44 [libx264 @ 0x7fe1d4bee500] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1d4bee500] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1d4bee500] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1d4bee500] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1d4bee500] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1d4bee500] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1d4bee500] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1d4bee500] 8x8 transform intra:0.0% [libx264 @ 0x7fe1d4bee500] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1d4bee500] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1d4bee500] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1d4bee500] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1d4bee500] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1d4bee500] kb/s:36.44 [libx264 @ 0x7fe1d40e32c0] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1d40e32c0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1d40e32c0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1d40e32c0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1d40e32c0] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1d40e32c0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1d40e32c0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1d40e32c0] 8x8 transform intra:0.0% [libx264 @ 0x7fe1d40e32c0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1d40e32c0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1d40e32c0] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1d40e32c0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1d40e32c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1d40e32c0] kb/s:36.44 [libx264 @ 0x7fe1d468e100] width not divisible by 2 (145x146) ERROR: encoder.c:261] Error opening video encoder : Generic error in an external library ERROR: transcoder.c:238] Unable to open output : Generic error in an external library E1215 19:25:42.626307 37054 ffmpeg.go:1012] Transcoder Return : Generic error in an external library ERROR: encoder.c:234] Unable to find encoder : Invalid argument ERROR: transcoder.c:238] Unable to open output : Invalid argument E1215 19:25:42.627582 37054 ffmpeg.go:1012] Transcoder Return : Invalid argument [libx264 @ 0x7fe1c802f740] width not divisible by 2 (145x146) ERROR: encoder.c:261] Error opening video encoder : Generic error in an external library ERROR: transcoder.c:238] Unable to open output : Generic error in an external library E1215 19:25:42.630150 37054 ffmpeg.go:1012] Transcoder Return : Generic error in an external library [libx264 @ 0x7fe1d4570940] width not divisible by 2 (145x146) ERROR: encoder.c:261] Error opening video encoder : Generic error in an external library ERROR: transcoder.c:238] Unable to open output : Generic error in an external library E1215 19:25:42.632730 37054 ffmpeg.go:1012] Transcoder Return : Generic error in an external library [libx264 @ 0x7fe1c8120dc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c8120dc0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c8a57cc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c8a57cc0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c93b05c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c93b05c0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c9d08e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c9d08e00] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c8120dc0] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1c8120dc0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c8120dc0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c8120dc0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c8120dc0] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1c8120dc0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c8120dc0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c8120dc0] 8x8 transform intra:0.0% [libx264 @ 0x7fe1c8120dc0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c8120dc0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c8120dc0] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1c8120dc0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1c8120dc0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c8120dc0] kb/s:36.44 [libx264 @ 0x7fe1c8a57cc0] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1c8a57cc0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c8a57cc0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c8a57cc0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c8a57cc0] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1c8a57cc0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c8a57cc0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c8a57cc0] 8x8 transform intra:0.0% [libx264 @ 0x7fe1c8a57cc0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c8a57cc0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c8a57cc0] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1c8a57cc0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1c8a57cc0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c8a57cc0] kb/s:36.44 [libx264 @ 0x7fe1c93b05c0] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1c93b05c0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c93b05c0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c93b05c0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c93b05c0] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1c93b05c0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c93b05c0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c93b05c0] 8x8 transform intra:0.0% [libx264 @ 0x7fe1c93b05c0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c93b05c0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c93b05c0] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1c93b05c0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1c93b05c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c93b05c0] kb/s:36.44 [libx264 @ 0x7fe1c9d08e00] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1c9d08e00] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c9d08e00] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c9d08e00] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c9d08e00] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1c9d08e00] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c9d08e00] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c9d08e00] 8x8 transform intra:0.0% [libx264 @ 0x7fe1c9d08e00] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c9d08e00] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c9d08e00] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1c9d08e00] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1c9d08e00] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c9d08e00] kb/s:36.44 [libx264 @ 0x7fe1c8a507c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c8a507c0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c9391dc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c9391dc0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c8d928c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c8d928c0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c934e5c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c934e5c0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c8a507c0] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1c8a507c0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c8a507c0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c8a507c0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c8a507c0] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1c8a507c0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c8a507c0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c8a507c0] 8x8 transform intra:0.0% [libx264 @ 0x7fe1c8a507c0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c8a507c0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c8a507c0] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1c8a507c0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1c8a507c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c8a507c0] kb/s:36.44 [libx264 @ 0x7fe1c9391dc0] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1c9391dc0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c9391dc0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c9391dc0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c9391dc0] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1c9391dc0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c9391dc0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c9391dc0] 8x8 transform intra:0.0% [libx264 @ 0x7fe1c9391dc0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c9391dc0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c9391dc0] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1c9391dc0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1c9391dc0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c9391dc0] kb/s:36.44 [libx264 @ 0x7fe1c8d928c0] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1c8d928c0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c8d928c0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c8d928c0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c8d928c0] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1c8d928c0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c8d928c0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c8d928c0] 8x8 transform intra:0.0% [libx264 @ 0x7fe1c8d928c0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c8d928c0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c8d928c0] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1c8d928c0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1c8d928c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c8d928c0] kb/s:36.44 [libx264 @ 0x7fe1c934e5c0] frame I:1 Avg QP: 0.00 size: 853 [libx264 @ 0x7fe1c934e5c0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c934e5c0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c934e5c0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c934e5c0] mb I I16..4: 100.0% 0.0% 0.0% [libx264 @ 0x7fe1c934e5c0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c934e5c0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c934e5c0] 8x8 transform intra:0.0% [libx264 @ 0x7fe1c934e5c0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c934e5c0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c934e5c0] i16 v,h,dc,p: 90% 0% 10% 0% [libx264 @ 0x7fe1c934e5c0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fe1c934e5c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c934e5c0] kb/s:36.44 [libx264 @ 0x7fe1c8a41880] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c8a41880] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1ca75b680] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1ca75b680] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1ca13d200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1ca13d200] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c8497440] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c8497440] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c8a41880] frame I:1 Avg QP: 0.00 size: 2818 [libx264 @ 0x7fe1c8a41880] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c8a41880] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c8a41880] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c8a41880] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1c8a41880] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c8a41880] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c8a41880] 8x8 transform intra:81.0% [libx264 @ 0x7fe1c8a41880] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c8a41880] coded y,uvDC,uvAC intra: 86.0% 12.0% 11.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c8a41880] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1c8a41880] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c8a41880] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c8a41880] i8c dc,h,v,p: 34% 64% 1% 1% [libx264 @ 0x7fe1c8a41880] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c8a41880] kb/s:115.04 [libx264 @ 0x7fe1ca75b680] frame I:1 Avg QP: 0.00 size: 2818 [libx264 @ 0x7fe1ca75b680] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1ca75b680] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1ca75b680] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1ca75b680] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1ca75b680] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1ca75b680] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1ca75b680] 8x8 transform intra:81.0% [libx264 @ 0x7fe1ca75b680] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1ca75b680] coded y,uvDC,uvAC intra: 86.0% 12.0% 11.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1ca75b680] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1ca75b680] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca75b680] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca75b680] i8c dc,h,v,p: 34% 64% 1% 1% [libx264 @ 0x7fe1ca75b680] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1ca75b680] kb/s:115.04 [libx264 @ 0x7fe1ca13d200] frame I:1 Avg QP: 0.00 size: 2818 [libx264 @ 0x7fe1ca13d200] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1ca13d200] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1ca13d200] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1ca13d200] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1ca13d200] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1ca13d200] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1ca13d200] 8x8 transform intra:81.0% [libx264 @ 0x7fe1ca13d200] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1ca13d200] coded y,uvDC,uvAC intra: 86.0% 12.0% 11.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1ca13d200] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1ca13d200] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca13d200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca13d200] i8c dc,h,v,p: 34% 64% 1% 1% [libx264 @ 0x7fe1ca13d200] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1ca13d200] kb/s:115.04 [libx264 @ 0x7fe1c8497440] frame I:1 Avg QP: 0.00 size: 2818 [libx264 @ 0x7fe1c8497440] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c8497440] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c8497440] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c8497440] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1c8497440] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c8497440] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c8497440] 8x8 transform intra:81.0% [libx264 @ 0x7fe1c8497440] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c8497440] coded y,uvDC,uvAC intra: 86.0% 12.0% 11.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c8497440] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1c8497440] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c8497440] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c8497440] i8c dc,h,v,p: 34% 64% 1% 1% [libx264 @ 0x7fe1c8497440] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c8497440] kb/s:115.04 [libx264 @ 0x7fe1c8a50d00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c8a50d00] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c9392880] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c9392880] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1ca78d040] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1ca78d040] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c9790cc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c9790cc0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c8a50d00] frame I:1 Avg QP: 0.00 size: 2818 [libx264 @ 0x7fe1c8a50d00] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c8a50d00] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c8a50d00] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c8a50d00] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1c8a50d00] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c8a50d00] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c8a50d00] 8x8 transform intra:81.0% [libx264 @ 0x7fe1c8a50d00] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c8a50d00] coded y,uvDC,uvAC intra: 86.0% 12.0% 11.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c8a50d00] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1c8a50d00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c8a50d00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c8a50d00] i8c dc,h,v,p: 34% 64% 1% 1% [libx264 @ 0x7fe1c8a50d00] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c8a50d00] kb/s:115.04 [libx264 @ 0x7fe1c9392880] frame I:1 Avg QP: 0.00 size: 2818 [libx264 @ 0x7fe1c9392880] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c9392880] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c9392880] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c9392880] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1c9392880] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c9392880] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c9392880] 8x8 transform intra:81.0% [libx264 @ 0x7fe1c9392880] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c9392880] coded y,uvDC,uvAC intra: 86.0% 12.0% 11.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c9392880] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1c9392880] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c9392880] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c9392880] i8c dc,h,v,p: 34% 64% 1% 1% [libx264 @ 0x7fe1c9392880] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c9392880] kb/s:115.04 [libx264 @ 0x7fe1ca78d040] frame I:1 Avg QP: 0.00 size: 2818 [libx264 @ 0x7fe1ca78d040] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1ca78d040] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1ca78d040] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1ca78d040] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1ca78d040] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1ca78d040] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1ca78d040] 8x8 transform intra:81.0% [libx264 @ 0x7fe1ca78d040] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1ca78d040] coded y,uvDC,uvAC intra: 86.0% 12.0% 11.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1ca78d040] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1ca78d040] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca78d040] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca78d040] i8c dc,h,v,p: 34% 64% 1% 1% [libx264 @ 0x7fe1ca78d040] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1ca78d040] kb/s:115.04 [libx264 @ 0x7fe1c9790cc0] frame I:1 Avg QP: 0.00 size: 2818 [libx264 @ 0x7fe1c9790cc0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c9790cc0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c9790cc0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c9790cc0] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1c9790cc0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c9790cc0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c9790cc0] 8x8 transform intra:81.0% [libx264 @ 0x7fe1c9790cc0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c9790cc0] coded y,uvDC,uvAC intra: 86.0% 12.0% 11.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c9790cc0] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1c9790cc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c9790cc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c9790cc0] i8c dc,h,v,p: 34% 64% 1% 1% [libx264 @ 0x7fe1c9790cc0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c9790cc0] kb/s:115.04 [libx264 @ 0x7fe1c800df00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1c800df00] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1ca663c00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1ca663c00] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1ca71d000] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1ca71d000] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1ca5d25c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe1ca5d25c0] profile High, level 2.0, 4:2:0, 8-bit [libx264 @ 0x7fe1c800df00] frame I:1 Avg QP: 0.00 size: 2723 [libx264 @ 0x7fe1c800df00] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1c800df00] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1c800df00] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1c800df00] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1c800df00] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1c800df00] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1c800df00] 8x8 transform intra:81.0% [libx264 @ 0x7fe1c800df00] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1c800df00] coded y,uvDC,uvAC intra: 86.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1c800df00] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1c800df00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c800df00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1c800df00] i8c dc,h,v,p: 28% 72% 0% 0% [libx264 @ 0x7fe1c800df00] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1c800df00] kb/s:111.24 [libx264 @ 0x7fe1ca663c00] frame I:1 Avg QP: 0.00 size: 2723 [libx264 @ 0x7fe1ca663c00] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1ca663c00] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1ca663c00] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1ca663c00] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1ca663c00] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1ca663c00] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1ca663c00] 8x8 transform intra:81.0% [libx264 @ 0x7fe1ca663c00] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1ca663c00] coded y,uvDC,uvAC intra: 86.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1ca663c00] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1ca663c00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca663c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca663c00] i8c dc,h,v,p: 28% 72% 0% 0% [libx264 @ 0x7fe1ca663c00] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1ca663c00] kb/s:111.24 [libx264 @ 0x7fe1ca71d000] frame I:1 Avg QP: 0.00 size: 2723 [libx264 @ 0x7fe1ca71d000] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1ca71d000] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1ca71d000] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1ca71d000] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1ca71d000] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1ca71d000] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1ca71d000] 8x8 transform intra:81.0% [libx264 @ 0x7fe1ca71d000] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1ca71d000] coded y,uvDC,uvAC intra: 86.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1ca71d000] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1ca71d000] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca71d000] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca71d000] i8c dc,h,v,p: 28% 72% 0% 0% [libx264 @ 0x7fe1ca71d000] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1ca71d000] kb/s:111.24 [libx264 @ 0x7fe1ca5d25c0] frame I:1 Avg QP: 0.00 size: 2723 [libx264 @ 0x7fe1ca5d25c0] frame P:1 Avg QP: 0.00 size: 16 [libx264 @ 0x7fe1ca5d25c0] frame B:3 Avg QP: 0.00 size: 14 [libx264 @ 0x7fe1ca5d25c0] consecutive B-frames: 20.0% 0.0% 0.0% 80.0% [libx264 @ 0x7fe1ca5d25c0] mb I I16..4: 11.0% 81.0% 8.0% [libx264 @ 0x7fe1ca5d25c0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fe1ca5d25c0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fe1ca5d25c0] 8x8 transform intra:81.0% [libx264 @ 0x7fe1ca5d25c0] direct mvs spatial:33.3% temporal:66.7% [libx264 @ 0x7fe1ca5d25c0] coded y,uvDC,uvAC intra: 86.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fe1ca5d25c0] i16 v,h,dc,p: 91% 0% 9% 0% [libx264 @ 0x7fe1ca5d25c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca5d25c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 75% 25% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe1ca5d25c0] i8c dc,h,v,p: 28% 72% 0% 0% [libx264 @ 0x7fe1ca5d25c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe1ca5d25c0] kb/s:111.24 --- FAIL: TestCapability_TranscoderCapabilities (0.32s) capabilities_test.go:150: Error Trace: /home/ricks/development/personal/go-livepeer/core/capabilities_test.go:150 Error: Expected nil, but got: &errors.errorString{s:"required capability \"H.264\" is not supported on hardware"} Test: TestCapability_TranscoderCapabilities 4 i:0 i:1 8ab313df2e516d86

Done
0000000000000000000000000000000000000000000000008ab313df2e516d86

0

E1215 19:25:45.856233 37054 orchestrator.go:594] Error transcoding segName=test.ts err="ErrTranscode"
E1215 19:25:45.856480 37054 orchestrator.go:600] Did not receive the correct number of transcoded segments; got 1 expected 2
E1215 19:25:47.870334 37054 orchestrator.go:848] Successfully received results from remote transcoder=TestAddress segments=1 taskId=1 fname= dur=3.924µs err="TranscodeError"
E1215 19:25:47.870372 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=2 fname= err="SendError"
E1215 19:25:47.875669 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=1001 fname= err="Remote transcoder took too long"
E1215 19:25:47.981063 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=1002 fname= err="Remote transcoder took too long"
E1215 19:25:47.992225 37054 orchestrator.go:848] Successfully received results from remote transcoder=TestAddress segments=1 taskId=1 fname= dur=2.837µs err="TranscodeError"
E1215 19:25:47.992277 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=2 fname= err="SendError"
E1215 19:25:47.992291 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=3 fname= err="SendError"
E1215 19:25:47.992300 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=4 fname= err="SendError"
E1215 19:25:47.992309 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=5 fname= err="SendError"
E1215 19:25:47.992318 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=6 fname= err="SendError"
E1215 19:25:47.992327 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=7 fname= err="SendError"
E1215 19:25:47.992336 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=8 fname= err="SendError"
E1215 19:25:47.992345 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=9 fname= err="SendError"
E1215 19:25:47.992356 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=10 fname= err="SendError"
E1215 19:25:47.992372 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=11 fname= err="SendError"
E1215 19:25:47.992382 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=12 fname= err="SendError"
E1215 19:25:47.992391 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=13 fname= err="SendError"
E1215 19:25:47.992403 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=14 fname= err="SendError"
E1215 19:25:47.992412 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=15 fname= err="SendError"
E1215 19:25:47.995364 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=16 fname= err="SendError"
E1215 19:25:47.998228 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=17 fname= err="SendError"
E1215 19:25:47.998282 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=18 fname= err="SendError"
E1215 19:25:47.998298 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=19 fname= err="SendError"
E1215 19:25:48.001629 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=20 fname= err="SendError"
E1215 19:25:48.001674 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=21 fname= err="SendError"
E1215 19:25:48.005066 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=22 fname= err="SendError"
E1215 19:25:48.007889 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=23 fname= err="SendError"
E1215 19:25:48.010636 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=24 fname= err="SendError"
E1215 19:25:48.013646 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=25 fname= err="SendError"
E1215 19:25:48.016767 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=26 fname= err="SendError"
E1215 19:25:48.016810 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=27 fname= err="SendError"
E1215 19:25:48.016822 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=28 fname= err="SendError"
E1215 19:25:48.016834 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=29 fname= err="SendError"
E1215 19:25:48.016847 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=30 fname= err="SendError"
E1215 19:25:48.019846 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=31 fname= err="SendError"
E1215 19:25:48.022705 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=32 fname= err="SendError"
E1215 19:25:48.022740 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=33 fname= err="SendError"
E1215 19:25:48.022763 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=34 fname= err="SendError"
E1215 19:25:48.022783 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=35 fname= err="SendError"
E1215 19:25:48.022798 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=36 fname= err="SendError"
E1215 19:25:48.029160 37054 orchestrator.go:804] Fatal error with remote transcoder=TestAddress taskId=37 fname= err="Remote transcoder took too long"
E1215 19:25:48.085979 37054 orchestrator.go:213] error redeeming ticket sessionID=some manifest recipientRandHash=44b6b308749d3fce761a79dae51e75b4a6f8d624a1d3e3f8176b670f4027cbb4 senderNonce=456 err="RedeemWinningTicket error"
E1215 19:25:48.180881 37054 orchestrator.go:185] Error receiving ticket sessionID=some manifest recipientRandHash=58b3bf533997d6dcbc6f1438fd89c64c42ebed067d2b1ff8a125d2c80e5d5f16 senderNonce=456: invalid ticket signature
E1215 19:25:48.202173 37054 orchestrator.go:185] Error receiving ticket sessionID=some manifest recipientRandHash=122756ee87716f72c9a541fd259a67616ca60bbba1c44eae8ab9f744831ab575 senderNonce=456: ReceiveTicket error
E1215 19:25:48.223011 37054 orchestrator.go:185] Error receiving ticket sessionID=some manifest recipientRandHash=ad710494b6f9cabbd76ac8918d582033e380cac1cbbd09b8788a87c3fbc80c1e senderNonce=456: ReceiveTicket error
E1215 19:25:48.244946 37054 orchestrator.go:185] Error receiving ticket sessionID=some manifest recipientRandHash=fb40d6fea7678387c5b4c09b300100c2a27c2383e74c1a76b30f3f862068114d senderNonce=456: ReceiveTicket error
E1215 19:25:49.419339 37054 transcoder.go:460] Cannot read transcoded output for name=badfile
E1215 19:25:49.419710 37054 transcoder.go:460] Cannot read transcoded output for name=badfile
[mpegts @ 0x7fe1e9035e80] frame size not set
FAIL
coverage: 75.7% of statements
FAIL github.com/livepeer/go-livepeer/core 19.642s
ok github.com/livepeer/go-livepeer/crypto 0.064s coverage: 90.0% of statements
ok github.com/livepeer/go-livepeer/discovery 6.230s coverage: 80.1% of statements
ok github.com/livepeer/go-livepeer/eth 33.804s coverage: 41.4% of statements
ok github.com/livepeer/go-livepeer/eth/blockwatch 0.314s coverage: 66.0% of statements
ok github.com/livepeer/go-livepeer/eth/types 0.132s coverage: 66.2% of statements
ok github.com/livepeer/go-livepeer/eth/watchers 1.700s coverage: 91.5% of statements
ok github.com/livepeer/go-livepeer/monitor 0.061s coverage: 54.7% of statements
ok github.com/livepeer/go-livepeer/pm 4.948s coverage: 83.5% of statements
ok github.com/livepeer/go-livepeer/server 24.656s coverage: 72.4% of statements
ok github.com/livepeer/go-livepeer/verification 5.478s coverage: 90.9% of statements
FAIL


</details>

@rickstaa
Copy link
Contributor Author

In the meantime, I created #2936 to run the test scripts on my production machine.

@rickstaa rickstaa mentioned this pull request Dec 16, 2023
Open
5 tasks
@rickstaa
Copy link
Contributor Author

@eliteprox I created an issue to enhance the Contribution guidelines using your notes. Feel free to leave your feedback at #2937.

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 16, 2023 •
edited
Loading

@leszko It looks like all tests pass. I will check whether the metric shows up correctly and request a new review when it does 👍🏻.

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 16, 2023 •
edited
Loading

@leszko, since the Livepeer contracts still need to be deployed on the goerli testnet, I was not able to test them without modifying the starter code (see #2664 (comment)). Can you guide me on how to test these metrics without using my mainnet wallet? I appreciate any help you can provide.

@rickstaa rickstaa marked this pull request as draft December 16, 2023 12:28
@rickstaa
Copy link
Contributor Author

@leszko, I created a small test script to test the pull request. Below is the code:

func TestRewardCallError(t *testing.T) {
	InitCensus("bctr", "testversion")
	census.nodeID = "0x5BdEedCA9c6346B0CE6b17ffa8227A4DACE37039"
	err := fmt.Errorf("some error")

	// Uncomment for direct error.
	RewardCallError(err.Error())

	// Uncomment for delayed error.
	// time.AfterFunc(10*time.Second, func() {
	// 	RewardCallError(err.Error())
	// })

	mux := http.DefaultServeMux
	mux.Handle("/metrics", Exporter)
	log.Fatal(http.ListenAndServe(":8936", mux))
}

I can confirm that this pull request is working as expected. However, I've converted this pull request to a draft as I believe it would be beneficial to provide more detailed information about the error, similar to what is done for the PaymentRecvError here: link to code.

I will mark it as ready for review once I've made these improvements. If you have any suggestions for a more efficient way to test my changes, please let me know, as it would significantly speed up my development process 🙏🏻.

@rickstaa
Copy link
Contributor Author

@leszko I already found https://github.com/livepeer/go-livepeer/blob/master/cmd/devtool/README.md 👍🏻.

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 17, 2023 •
edited
Loading

@eliteprox, @leszko, I've completed several updates to the reward_call_errors metric in this pull request, and it's now ready for merging. However, there are two opportunities to improve this pull request:

  • Catch more errors: Also catching errors of on-chain calls will give users more information about why a transaction failed.
  • Add the transaction hash as a label when we receive a TransactionFailed or TransactionTimeout error: This will give users information about which transaction failed.

Please take a moment to review the changes and consider the possible enhancements we can implement. Feel free to share further suggestions or let me know if you can proceed with the merge.

Uncaught errors

GetTranscoder

I've identified various errors that can occur when calling the GetTranscoder client method. I've now consolidated these errors under the RewardCallError error code.

go-livepeer/eth/client.go

Lines 800 to 844 in 002e602

func (c *client) GetTranscoder(addr ethcommon.Address) (*lpTypes.Transcoder, error) {
tInfo, err := c.bondingManager.GetTranscoder(c.callOpts(), addr)
if err != nil {
return nil, err
}
tStatus, err := c.TranscoderStatus(addr)
if err != nil {
return nil, err
}
status, err := lpTypes.ParseTranscoderStatus(tStatus)
if err != nil {
return nil, err
}
delegatedStake, err := c.TranscoderTotalStake(addr)
if err != nil {
return nil, err
}
active, err := c.bondingManager.IsActiveTranscoder(c.callOpts(), addr)
if err != nil {
return nil, err
}
serviceURI, err := c.GetServiceURI(addr)
if err != nil {
return nil, err
}
return &lpTypes.Transcoder{
Address: addr,
ServiceURI: serviceURI,
LastRewardRound: tInfo.LastRewardRound,
RewardCut: tInfo.RewardCut,
FeeShare: tInfo.FeeShare,
DelegatedStake: delegatedStake,
ActivationRound: tInfo.ActivationRound,
DeactivationRound: tInfo.DeactivationRound,
LastActiveStakeUpdateRound: tInfo.LastActiveStakeUpdateRound,
Active: active,
Status: status,
}, nil
}

s.client.Reward()

Similarly, I've identified several errors when invoking the s.client.Reward() client method, which I've now grouped under the RewardCallError error code.

go-livepeer/eth/client.go

Lines 989 to 1035 in 002e602

func (c *client) Reward() (*types.Transaction, error) {
addr := c.accountManager.Account().Address
tr, err := c.GetTranscoder(addr)
if err != nil {
return nil, err
}
ep, err := c.GetTranscoderEarningsPoolForRound(addr, tr.LastActiveStakeUpdateRound)
if err != nil {
return nil, err
}
activeTotalStake := ep.TotalStake
mintable, err := c.CurrentMintableTokens()
if err != nil {
return nil, errors.Wrapf(err, "unable to get current mintable tokens")
}
totalBonded, err := c.GetTotalBonded()
if err != nil {
return nil, errors.Wrapf(err, "unable to get total bonded")
}
if totalBonded.Cmp(big.NewInt(0)) == 0 {
return nil, errors.New("no rewards to be minted")
}
// reward = (current mintable tokens for the round * active transcoder stake) / total active stake
reward := new(big.Int).Div(new(big.Int).Mul(mintable, activeTotalStake), totalBonded)
// get the transcoder pool
transcoders, err := c.TranscoderPool()
if err != nil {
return nil, errors.Wrapf(err, "unable to get transcoder pool")
}
// get max pool size
maxSize, err := c.GetTranscoderPoolMaxSize()
if err != nil {
return nil, errors.Wrapf(err, "unable to get transcoder pool max size")
}
hints := simulateTranscoderPoolUpdate(addr, reward.Add(reward, tr.DelegatedStake), transcoders, len(transcoders) == int(maxSize.Int64()))
return c.bondingManager.RewardWithHint(c.transactOpts(), hints.PosPrev, hints.PosNext)
}

s.client.CheckTx(tx)

Regarding the s.client.CheckTx(tx) client method, I've identified several errors that were not properly handled and included them under the RewardCallError error code.

go-livepeer/eth/client.go

Lines 1109 to 1135 in 002e602

func (c *client) CheckTx(tx *types.Transaction) error {
receipts := make(chan *transactionReceipt, 10)
txSub := c.tm.Subscribe(receipts)
defer txSub.Unsubscribe()
timer := time.NewTimer(c.checkTxTimeout)
defer timer.Stop()
for {
select {
case <-timer.C:
return fmt.Errorf("timed out waiting for transaction receipt txHash=%v", tx.Hash().Hex())
case err := <-txSub.Err():
return err
case receipt := <-receipts:
if tx.Hash() == receipt.originTxHash {
if receipt.err != nil {
return receipt.err
}
if receipt.Status == uint64(0) {
return fmt.Errorf("transaction failed txHash=%v", receipt.TxHash.Hex())
}
return nil
}
}
}
}

  • I've addressed the missing error handling for this line.
  • Additionally, I've now caught errors that can arise from this line.

@rickstaa rickstaa marked this pull request as ready for review December 17, 2023 10:23
@rickstaa
Copy link
Contributor Author

@eliteprox, @leszko, I added 772806c to add the transaction hash to the reward call error metric. This label can be helpful when a transaction is sent but fails. Feel free to ask me to revert this commit if you think this situation will not occur. Otherwise, this pull request can be merged if you don't want to catch other error codes 😃.

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 18, 2023 •
edited
Loading

Oh, wait, it looks like the tests now fail (i.e. https://github.com/livepeer/go-livepeer/actions/runs/7245228816/job/19734872448?pr=2905) with the following error:

E1218 08:07:36.161598   11791 rewardservice.go:60] Error trying to call reward err="context deadline exceeded"
panic: runtime error: invalid memory address or nil pointer dereference

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 18, 2023 •
edited
Loading

Hello @leszko,

I've been investigating the test failure and implemented a panic recovery mechanism to handle potential panic while attempting to retrieve the transaction hash. The modified code snippet is shown below:

-var txHash string
+var txHash string
 if tx != nil {
+    defer func() {
+        if r := recover(); r != nil {
+            fmt.Println(tx)
+            fmt.Println("Recovered in tx.Hash():", r)
+            txHash = ""
+        }
+    }()
+    txHash = tx.Hash().Hex()
 }

However, I'd like to point out that the issue may not be expected because, in theory, the Hash method of *types.Transaction should always return a valid common.Hash object. Even if the hash field of the Transaction struct is nil, the common.Hash type is a fixed-size array, not a pointer, so it can't be nil. You can review the implementation here.

I therefore am unsure why this issue occurs. I noticed a similar problem documented in a previous issue:

go-livepeer/eth/transactionManager_test.go

Lines 130 to 135 in 8a8b0e3

// Test error
// This calls bind.WaitMined() on the ethereum client, which will never actually return the actual underlying error and only log it using go-ethereum's custom logger
// The expected error should thus be 'context deadline exceeded'
// https://github.com/ethereum/go-ethereum/blob/aa637fd38a379db6da98df0d520fb1c5139a18ce/accounts/abi/bind/util.go#L41
expErr := errors.New("context deadline exceeded")
eth.err["TransactionByHash"] = expErr

I would greatly appreciate your input if you have any insights or guidance on addressing this issue or if there's anything else I should check. Thank you in advance!

@rickstaa rickstaa mentioned this pull request Dec 18, 2023
Open
@rickstaa rickstaa mentioned this pull request Dec 18, 2023
Closed
5 tasks
@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 18, 2023 •
edited
Loading

@leszko I have found another potential solution to the issue, which involves modifying the code as shown below:

-if tx != nil {
-    txHash = tx.Hash().String()
-}
+if tx != nil && !strings.Contains(err.Error(), "context deadline exceeded") {
+    txHash = tx.Hash().Hex()
+}

Since I need to fully understand why the Hash() method is not always defined.

@rickstaa
Copy link
Contributor Author

Hey @leszko, I've implemented the solution as outlined in commit 167cbf3. However, I would like to emphasize the importance of reviewing this pull request thoroughly. Unfortunately, I couldn't pinpoint the exact root cause that necessitated the changes in commit 167cbf3, so we should be extra cautious in our assessment 😅.

@rickstaa
feat(monitor): add reward service error metric
3bff611 
This commit adds the `reward_call_errors` prometheus metric when users
enable the monitor function. This allows users to see when a reward call
failed.
@rickstaa
refactor: add CHANGELOG_PENDING entry
ad61d75
@rickstaa
chore: improve 'CHANGELOG_PENDING' entry
46a33d1
@rickstaa rickstaa force-pushed the add_reward_service_error_monitor_parameter branch from 167cbf3 to 923375c Compare December 18, 2023 13:46
leszko
leszko reviewed Dec 19, 2023
View reviewed changes
Copy link
Contributor

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for the PR @rickstaa

I added one inline comment. Plus, could you update the PR description with how you tested this change? Thanks.

eth/rewardservice.go Outdated
if err != nil {
glog.Errorf("Error trying to call reward err=%q", err)
var txHash string
if tx != nil && !strings.Contains(err.Error(), "context deadline exceeded") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you move "parsing" the error to monitor.RewardCallError()? I think you can pass there the whole tx and err. I'd try to avoid parsing the error in multiple places.

@rickstaa
Copy link
Contributor Author

rickstaa commented Dec 19, 2023 •
edited
Loading

@leszko, the tests are passing locally but still failing in CI (details in CI run #19782520969) 😅. This discrepancy suggests a potential race condition. As I mentioned in my comment, the issue might relate to the tx.hash atomic.Value not being correctly set. However, this is perplexing because the common.Hash type, being a fixed-size array, shouldn't be nil even if the hash field of the Transaction struct is 🤔. I'm at an impasse due to my limited familiarity with the codebase, so your insights and assistance in resolving this for the pull request would be very much appreciated.

@leszko
Copy link
Contributor

leszko commented Dec 19, 2023

@leszko, the tests are passing locally but still failing in CI (details in CI run #19782520969) 😅. This discrepancy suggests a potential race condition. As I mentioned in my comment, the issue might relate to the tx.hash atomic.Value not being correctly set. However, this is perplexing because the common.Hash type, being a fixed-size array, shouldn't be nil even if the hash field of the Transaction struct is 🤔. I'm at an impasse due to my limited familiarity with the codebase, so your insights and assistance in resolving this for the pull request would be very much appreciated.

I've re-run the tests, maybe it's just some flakiness. We have a couple of flaky tests 🙈

When I asked about testing and updating the description I meant more like some manual tests if the metric you added works. How did you test it? Could you add a few words to the description?

@rickstaa
Copy link
Contributor Author

@leszko, the tests are passing locally but still failing in CI (details in CI run #19782520969) 😅. This discrepancy suggests a potential race condition. As I mentioned in my comment, the issue might relate to the tx.hash atomic.Value not being correctly set. However, this is perplexing because the common.Hash type, being a fixed-size array, shouldn't be nil even if the hash field of the Transaction struct is 🤔. I'm at an impasse due to my limited familiarity with the codebase, so your insights and assistance in resolving this for the pull request would be very much appreciated.

I've re-run the tests, maybe it's just some flakiness. We have a couple of flaky tests 🙈

When I asked about testing and updating the description I meant more like some manual tests if the metric you added works. How did you test it? Could you add a few words to the description?

That could be why the tests throw many warnings on my local machine (see log below). I, however, want to be sure before deploying it to the main branch and, therefore, want to understand the problem. To test it, I'm currently using the following workflow:

  1. Run the tests to see if they pass.
  2. Spin a private Ethereum dev network according to this guide.
  3. Add the following launch.json to my .vscode folder.
{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Attach to Process",
      "type": "go",
      "request": "attach",
      "mode": "local",
      "processId": 0
    },
    {
      "name": "Launch Package",
      "type": "go",
      "request": "launch",
      "mode": "auto",
      "program": "${fileDirname}",
      "args": [
        "-v",
        "99",
        "-ethController",
        "0x369ab95e343a2ff18e241a4b48e2ff2d9de1fdf3",
        "-dataDir",
        "../../.lpdev2/orchestrator_<DEV_NET_WALLET>",
        "-ethAcctAddr",
        "<DEV_NET_WALLET>",
        "-ethUrl",
        "http://localhost:8545/",
        "-ethPassword",
        "",
        "-network",
        "devenv",
        "-blockPollingInterval",
        "1",
        "-monitor",
        "-currentManifest",
        "-cliAddr",
        "127.0.0.1:7937",
        "-httpAddr",
        "127.0.0.1:8936",
        "-initializeRound=true",
        "-serviceAddr",
        "127.0.0.1:8936",
        "-orchestrator",
        "-orchSecret",
        "secre",
        "-pricePerUnit",
        "1",
        "-transcoder",
        "-reward",
      ]
    },
  ]
}
  1. Manually throw different errors using err = fmt.Errorf("<ERROR_MESSAGE>") after the following code:

go-livepeer/eth/rewardservice.go

Line 58 in c4b6153

tx, err := s.tryReward()

  1. Check if the script does not panic and if the error appears correctly on https://localhost:7937/metrics.

Logs

Test logs 
++ go list ./...
++ grep -v test/e2e
+ go test -coverprofile cover.out github.com/livepeer/go-livepeer/build github.com/livepeer/go-livepeer/clog github.com/livepeer/go-livepeer/cmd/devtool github.com/livepeer/go-livepeer/cmd/devtool/devtool github.com/livepeer/go-livepeer/cmd/livepeer github.com/livepeer/go-livepeer/cmd/livepeer/starter github.com/livepeer/go-livepeer/cmd/livepeer_bench github.com/livepeer/go-livepeer/cmd/livepeer_cli github.com/livepeer/go-livepeer/cmd/livepeer_router github.com/livepeer/go-livepeer/cmd/scripts github.com/livepeer/go-livepeer/cmd/simple_auth_server github.com/livepeer/go-livepeer/common github.com/livepeer/go-livepeer/core github.com/livepeer/go-livepeer/crypto github.com/livepeer/go-livepeer/discovery github.com/livepeer/go-livepeer/eth github.com/livepeer/go-livepeer/eth/blockwatch github.com/livepeer/go-livepeer/eth/contracts github.com/livepeer/go-livepeer/eth/types github.com/livepeer/go-livepeer/eth/watchers github.com/livepeer/go-livepeer/monitor github.com/livepeer/go-livepeer/net github.com/livepeer/go-livepeer/pm github.com/livepeer/go-livepeer/server github.com/livepeer/go-livepeer/verification
?       github.com/livepeer/go-livepeer/build   [no test files]
ok      github.com/livepeer/go-livepeer/clog    0.009s  coverage: 69.5% of statements
?       github.com/livepeer/go-livepeer/cmd/devtool     [no test files]
?       github.com/livepeer/go-livepeer/cmd/devtool/devtool     [no test files]
?       github.com/livepeer/go-livepeer/cmd/livepeer_bench      [no test files]
?       github.com/livepeer/go-livepeer/cmd/livepeer_router     [no test files]
?       github.com/livepeer/go-livepeer/cmd/scripts     [no test files]
?       github.com/livepeer/go-livepeer/cmd/simple_auth_server  [no test files]
ok      github.com/livepeer/go-livepeer/cmd/livepeer    0.044s  coverage: 55.5% of statements
ok      github.com/livepeer/go-livepeer/cmd/livepeer/starter    0.051s  coverage: 5.4% of statements
ok      github.com/livepeer/go-livepeer/cmd/livepeer_cli        0.048s  coverage: 0.6% of statements
ok      github.com/livepeer/go-livepeer/common  0.099s  coverage: 66.6% of statements
?       github.com/livepeer/go-livepeer/eth/contracts   [no test files]
?       github.com/livepeer/go-livepeer/net     [no test files]
ok      github.com/livepeer/go-livepeer/core    21.306s coverage: 76.4% of statements
ok      github.com/livepeer/go-livepeer/crypto  0.021s  coverage: 90.0% of statements
ok      github.com/livepeer/go-livepeer/discovery       6.184s  coverage: 80.6% of statements
ok      github.com/livepeer/go-livepeer/eth     32.710s coverage: 41.4% of statements
ok      github.com/livepeer/go-livepeer/eth/blockwatch  0.269s  coverage: 66.0% of statements
ok      github.com/livepeer/go-livepeer/eth/types       0.021s  coverage: 66.2% of statements
ok      github.com/livepeer/go-livepeer/eth/watchers    1.615s  coverage: 91.5% of statements
ok      github.com/livepeer/go-livepeer/monitor 0.046s  coverage: 53.3% of statements
ok      github.com/livepeer/go-livepeer/pm      4.871s  coverage: 83.5% of statements
ok      github.com/livepeer/go-livepeer/server  24.545s coverage: 72.4% of statements
ok      github.com/livepeer/go-livepeer/verification    3.866s  coverage: 90.9% of statements
+ cd core
+ go test -run LB_ -race
PASS
ok      github.com/livepeer/go-livepeer/core    1.152s
+ go test -run Nvidia_ -race
PASS
ok      github.com/livepeer/go-livepeer/core    1.082s
+ go test -run Capabilities_ -race
PASS
ok      github.com/livepeer/go-livepeer/core    1.078s
+ cd ..
+ cd discovery
+ go test -race
E1219 15:01:59.347482   86086 discovery.go:39] Orchestrator pool does not have any URIs
E1219 15:02:02.352744   86086 db_discovery.go:333] missing ETH address orch=transcoder
E1219 15:02:02.352886   86086 db_discovery.go:333] missing ETH address orch=transcoder
E1219 15:02:02.358149   86086 db_discovery.go:333] missing ETH address orch=transcoder
E1219 15:02:02.358924   86086 db_discovery.go:333] missing ETH address orch=transcoder
E1219 15:02:02.361390   86086 db_discovery.go:333] missing ETH address orch=transcoder
E1219 15:02:02.471266   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:02.471319   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:02.474809   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:02.477805   86086 db_discovery.go:333] Could not parse orchestrator URI: parse "https://badUrl\\://127.0.0.1:8936": invalid character "\\" in host name
E1219 15:02:02.580730   86086 db_discovery.go:333] missing ETH address orch=transcoderfromtestserver
E1219 15:02:02.580812   86086 db_discovery.go:333] missing ETH address orch=transcoderfromtestserver
E1219 15:02:02.583746   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:02.583781   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:02.583809   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.797177   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.797305   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.800967   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.803595   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.803664   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.806284   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.806340   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.808952   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.811406   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.813914   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.813973   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.816597   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.819167   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.821823   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.824424   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.827044   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.827114   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.830214   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.832857   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.832907   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.835525   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.838145   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.838211   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.840946   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.843523   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.846242   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.846305   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.849274   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.851845   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.854544   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.857148   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.857252   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.859888   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.862540   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.862600   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.865361   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.867967   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.868072   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.870819   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.873417   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.876022   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.878639   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.881181   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.883712   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.883754   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.886436   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.889042   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.889110   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.891801   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:03.894423   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.010503   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.010572   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.010600   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.010624   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.010649   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.014081   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.016664   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.019335   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.019396   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.022167   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.022236   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.025019   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.027635   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.030474   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.030523   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.033213   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.035807   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.035843   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.035867   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.038484   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.041026   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.041068   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.043676   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.046216   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.048705   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.048732   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.048757   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.051383   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.053928   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.056409   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.059041   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.061750   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.061822   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.064509   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.067110   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.067156   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.069847   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.072419   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.075078   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.077720   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.077786   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.080749   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.083383   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.086158   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.088899   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.091547   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.094205   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.096978   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.099738   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.102321   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.220798   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.220842   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.220864   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.223568   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.226093   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.228587   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.228612   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.228641   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.232463   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.237339   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.239806   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.239850   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.243105   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.247216   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.247255   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.252220   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.257811   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.257863   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.263575   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.272361   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.278024   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.283625   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.283681   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.289410   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.295137   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.297707   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.300205   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.300248   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.302902   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.305418   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.305446   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.305469   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.307994   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.310497   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.313012   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.313052   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.313066   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.315669   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.318208   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.318250   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.320883   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.323405   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.323445   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.326110   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.328744   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.328808   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.331641   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.334123   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.334169   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.336798   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.454387   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.454431   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.454451   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.454471   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.457190   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.459716   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.459761   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.462271   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.464834   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.464871   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.467486   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.470012   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.472531   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.475128   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.477608   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.477664   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.480163   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.482712   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.485235   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.487696   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.490220   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.490253   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.490277   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.492760   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.495215   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.495245   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.495264   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.497872   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.500283   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.502806   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.505310   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.507823   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.507847   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.507867   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.510504   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.513072   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.513111   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.513134   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.515668   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.518216   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.520709   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.520751   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.523410   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.523445   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.523475   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.526117   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.528623   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.531100   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.533694   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.536611   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.566684   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.566761   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.569680   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.569736   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.572432   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.575048   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.577635   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.577689   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.580224   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.580275   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.582864   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.585322   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.587894   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.590465   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.593249   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.593305   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.595905   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.595946   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.598625   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.601259   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.603831   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.606458   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.609056   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.609117   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.611660   86086 db_discovery.go:231] reference to rat is nil
E1219 15:02:04.721482   86086 wh_discovery.go:143] Unable to unmarshal JSON unexpected end of JSON input
E1219 15:02:04.721539   86086 wh_discovery.go:143] Unable to unmarshal JSON invalid character '\x00' looking for beginning of value
E1219 15:02:04.721577   86086 wh_discovery.go:143] Unable to unmarshal JSON json: cannot unmarshal object into Go value of type []discovery.webhookResponse
E1219 15:02:04.721602   86086 wh_discovery.go:143] Unable to unmarshal JSON json: cannot unmarshal number into Go value of type []discovery.webhookResponse
PASS
ok      github.com/livepeer/go-livepeer/discovery       7.673s
+ cd ..
+ cd server
+ go test -run TestSelectSession_ -race
PASS
ok      github.com/livepeer/go-livepeer/server  5.305s
+ go test -run RegisterConnection -race
E1219 15:02:14.886401   87076 mediaserver.go:504] Missing node storage
PASS
ok      github.com/livepeer/go-livepeer/server  1.142s
+ cd ..
+ ./test_args.sh
+ TMPDIR=/home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057
+ DEFAULT_DATADIR=/home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/.lpData
+ CUSTOM_DATADIR=/home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir
+ rm -rf /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/.lpData
+ mkdir -p /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057
+ HIGHEST_CHAIN_TAG=mainnet
+ go build -tags mainnet -o /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer cmd/livepeer/livepeer.go cmd/livepeer/livepeer_test.go
+ cat
+ ETH_ARGS='-ethKeystorePath /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/key'
+ SLEEP=0.25
+ export HOME=/home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057
+ HOME=/home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057
+ '[' '!' -d /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/.lpData ']'
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer
*
*
I1219 15:02:19.460360   87329 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:19.460425   87329 starter.go:424] Creating data dir: /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/.lpData/offchain
E1219 15:02:19.512481   87329 starter.go:1588] No services enabled; must be at least one of -broadcaster, -transcoder, -orchestrator, -redeemer, -reward or -initializeRound
+ res=2
+ '[' 2 -ne 0 ']'
+ run_lp -broadcaster
+ pid=87343
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster
*-------------*------*
| Broadcaster | true |
*-------------*------*
I1219 15:02:19.537546   87343 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:19.538329   87343 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:19.543781   87343 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:19.543851   87343 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:19.543876   87343 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:19.543897   87343 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:19.543975   87343 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ '[' -d /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/.lpData/offchain ']'
+ kill 87343
+ '[' '!' -d /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir ']'
+ run_lp -broadcaster -dataDir /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir
+ pid=87356
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -dataDir /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir
*-------------*------------------------------------------------------------------------------------*
| Broadcaster | true                                                                               |
| Datadir     | /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir |
*-------------*------------------------------------------------------------------------------------*
I1219 15:02:19.789369   87356 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:19.789419   87356 starter.go:424] Creating data dir: /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir
I1219 15:02:19.839572   87356 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:19.845018   87356 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:19.845092   87356 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:19.845119   87356 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:19.845141   87356 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:19.845241   87356 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ '[' -d /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir ']'
+ '[' '!' -d /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir/offchain ']'
+ kill 87356
+ CUSTOM_DATADIR=/home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir2
+ '[' '!' -d /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/customDatadir2 ']'
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -orchestrator -orchSecret asdf -serviceAddr $'hi\b\bbye'
+ grep 'Error getting service URI'
F1219 15:02:20.044201   87370 starter.go:1152] Error getting service URI: parse "https://hi\b\bbye": net/url: invalid control character in URL
+ '[' 1 -ne 0 ']'
+ run_lp -orchestrator -serviceAddr 127.0.0.1:8937 -transcoder
+ pid=87384
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -orchestrator -serviceAddr 127.0.0.1:8937 -transcoder
*--------------*----------------*
| ServiceAddr  | 127.0.0.1:8937 |
| Orchestrator | true           |
| Transcoder   | true           |
*--------------*----------------*
I1219 15:02:20.068642   87384 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.069445   87384 starter.go:565] ***Livepeer is in off-chain mode***
I1219 15:02:20.069502   87384 starter.go:1262] ***Livepeer Running in Orchestrator Mode***
I1219 15:02:20.069521   87384 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:20.069584   87384 cert.go:83] Private key and cert not found. Generating
I1219 15:02:20.069612   87384 webserver.go:20] CLI server listening on 127.0.0.1:7935
I1219 15:02:20.069851   87384 cert.go:22] Generating cert for 127.0.0.1
I1219 15:02:20.070092   87384 rpc.go:205] Listening for RPC on :8937
+ kill 87384
+ run_lp -orchestrator -serviceAddr 127.0.0.1:8937 -orchSecret asdf
+ pid=87397
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -orchestrator -serviceAddr 127.0.0.1:8937 -orchSecret asdf
*--------------*----------------*
| ServiceAddr  | 127.0.0.1:8937 |
| Orchestrator | true           |
| OrchSecret   | asdf           |
*--------------*----------------*
I1219 15:02:20.321577   87397 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.322350   87397 starter.go:565] ***Livepeer is in off-chain mode***
I1219 15:02:20.322408   87397 starter.go:1262] ***Livepeer Running in Orchestrator Mode***
I1219 15:02:20.322427   87397 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:20.322484   87397 cert.go:83] Private key and cert not found. Generating
I1219 15:02:20.322498   87397 webserver.go:20] CLI server listening on 127.0.0.1:7935
I1219 15:02:20.322797   87397 cert.go:22] Generating cert for 127.0.0.1
I1219 15:02:20.323051   87397 rpc.go:205] Listening for RPC on :8937
+ kill 87397
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -orchestrator -serviceAddr 127.0.0.1:8937
*--------------*----------------*
| ServiceAddr  | 127.0.0.1:8937 |
| Orchestrator | true           |
*--------------*----------------*
I1219 15:02:20.574225   87412 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.575164   87412 starter.go:565] ***Livepeer is in off-chain mode***
F1219 15:02:20.575209   87412 starter.go:1159] Running an orchestrator requires an -orchSecret for standalone mode or -transcoder for orchestrator+transcoder mode
+ res=1
+ '[' 1 -ne 0 ']'
+ '[' -z ']'
+ echo 'MAINNET_ETH_URL is not set - skipping mainnet tests'
MAINNET_ETH_URL is not set - skipping mainnet tests
+ '[' -z ']'
+ echo 'RINKEBY_ETH_URL is not set - skipping Rinkeby tests'
RINKEBY_ETH_URL is not set - skipping Rinkeby tests
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -transcoder
*------------*------*
| Transcoder | true |
*------------*------*
I1219 15:02:20.599836   87426 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.600940   87426 starter.go:565] ***Livepeer is in off-chain mode***
F1219 15:02:20.601004   87426 starter.go:1251] Missing -orchSecret
+ res=1
+ '[' 1 -ne 0 ']'
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -authWebhookUrl tcp://host/
*----------------*-------------*
| Broadcaster    | true        |
| AuthWebhookURL | tcp://host/ |
*----------------*-------------*
I1219 15:02:20.625496   87441 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.626283   87441 starter.go:565] ***Livepeer is in off-chain mode***
F1219 15:02:20.626320   87441 starter.go:1040] Error setting auth webhook URL URL should be HTTP or HTTPS
+ res=1
+ '[' 1 -ne 0 ']'
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -authWebhookUrl 'http\://host/'
*----------------*---------------*
| Broadcaster    | true          |
| AuthWebhookURL | http\://host/ |
*----------------*---------------*
I1219 15:02:20.650601   87456 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.651479   87456 starter.go:565] ***Livepeer is in off-chain mode***
F1219 15:02:20.651515   87456 starter.go:1040] Error setting auth webhook URL parse "http\\://host/": invalid URI for request
+ res=1
+ '[' 1 -ne 0 ']'
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -orchWebhookUrl tcp://host/
*----------------*-------------*
| Broadcaster    | true        |
| OrchWebhookURL | tcp://host/ |
*----------------*-------------*
I1219 15:02:20.675847   87470 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.676754   87470 starter.go:565] ***Livepeer is in off-chain mode***
F1219 15:02:20.676801   87470 starter.go:1089] Error setting orch webhook URL URL should be HTTP or HTTPS
+ res=1
+ '[' 1 -ne 0 ']'
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -orchWebhookUrl 'http\://host/'
*----------------*---------------*
| Broadcaster    | true          |
| OrchWebhookURL | http\://host/ |
*----------------*---------------*
I1219 15:02:20.700746   87485 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.701616   87485 starter.go:565] ***Livepeer is in off-chain mode***
F1219 15:02:20.701653   87485 starter.go:1089] Error setting orch webhook URL parse "http\\://host/": invalid URI for request
+ res=1
+ '[' 1 -ne 0 ']'
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -maxSessions -1
*-------------*------*
| Broadcaster | true |
| MaxSessions | -1   |
*-------------*------*
F1219 15:02:20.726077   87500 starter.go:338] -maxSessions must be 'auto' or greater than zero
+ res=1
+ '[' 1 -ne 0 ']'
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -maxSessions 0
*-------------*------*
| Broadcaster | true |
| MaxSessions | 0    |
*-------------*------*
F1219 15:02:20.749733   87515 starter.go:338] -maxSessions must be 'auto' or greater than zero
+ res=1
+ '[' 1 -ne 0 ']'
+ run_lp -broadcaster
+ pid=87529
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster
*-------------*------*
| Broadcaster | true |
*-------------*------*
I1219 15:02:20.773768   87529 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:20.774563   87529 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:20.774600   87529 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:20.774643   87529 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:20.774659   87529 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:20.774673   87529 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:20.774754   87529 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -sI http://127.0.0.1:5935/debug/pprof/allocs
+ grep '200 OK'
HTTP/1.1 200 OK
+ kill 87529
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -verifierUrl tcp://host/
*-------------*-------------*
| VerifierURL | tcp://host/ |
| Broadcaster | true        |
*-------------*-------------*
I1219 15:02:21.044012   87546 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:21.044972   87546 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:21.045016   87546 starter.go:1099] No orchestrator specified; transcoding will not happen
F1219 15:02:21.045040   87546 starter.go:1128] Error setting verifier URL URL should be HTTP or HTTPS
+ res=1
+ '[' 1 -ne 0 ']'
+ res=0
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -verifierUrl 'http\://host/'
*-------------*---------------*
| VerifierURL | http\://host/ |
| Broadcaster | true          |
*-------------*---------------*
I1219 15:02:21.069249   87560 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:21.070207   87560 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:21.070243   87560 starter.go:1099] No orchestrator specified; transcoding will not happen
F1219 15:02:21.070264   87560 starter.go:1128] Error setting verifier URL parse "http\\://host/": invalid URI for request
+ res=1
+ '[' 1 -ne 0 ']'
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -verifierUrl http://host
+ grep 'Requires a path to the'
F1219 15:02:21.095487   87575 starter.go:1136] Requires a path to the verifier shared volume when local storage is in use; use -verifierPath or -objectStore
+ run_lp -broadcaster -verifierUrl http://host -verifierPath path
+ pid=87589
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -verifierUrl http://host -verifierPath path
*--------------*-------------*
| VerifierURL  | http://host |
| VerifierPath | path        |
| Broadcaster  | true        |
*--------------*-------------*
I1219 15:02:21.119584   87589 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:21.120481   87589 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:21.120517   87589 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:21.120555   87589 starter.go:1130] Using the Epic Labs classifier for verification at http://host
I1219 15:02:21.120598   87589 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:21.120623   87589 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:21.120637   87589 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:21.120722   87589 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ kill 87589
+ run_lp -broadcaster -verifierUrl http://host -objectStore s3+https://ACCESS_KEY_ID:[email protected]/bucket-name
+ pid=87602
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -verifierUrl http://host -objectStore s3+https://ACCESS_KEY_ID:[email protected]/bucket-name
*-------------*----------------------------------------------------------------------------*
| VerifierURL | http://host                                                                |
| Broadcaster | true                                                                       |
| Objectstore | s3+https://ACCESS_KEY_ID:[email protected]/bucket-name |
*-------------*----------------------------------------------------------------------------*
I1219 15:02:21.372355   87602 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:21.373288   87602 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:21.373416   87602 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:21.373436   87602 starter.go:1130] Using the Epic Labs classifier for verification at http://host
I1219 15:02:21.373496   87602 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:21.373519   87602 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:21.373544   87602 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:21.373611   87602 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ kill 87602
+ run_lp -broadcaster -httpAddr 0.0.0.0
+ pid=87617
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -httpAddr 0.0.0.0
*-------------*---------*
| HttpAddr    | 0.0.0.0 |
| Broadcaster | true    |
*-------------*---------*
I1219 15:02:21.624466   87617 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:21.625274   87617 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:21.625314   87617 starter.go:1099] No orchestrator specified; transcoding will not happen
W1219 15:02:21.625334   87617 starter.go:1111] HTTP ingest is disabled because -httpAddr is publicly accessible. To enable, configure -authWebhookUrl or use the -httpIngest flag
I1219 15:02:21.625376   87617 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:21.625391   87617 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:21.625405   87617 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:21.625469   87617 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -X PUT http://localhost:9935/live/movie/0.ts
+ grep '404 page not found'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    19  100    19    0     0  30254      0 --:--:-- --:--:-- --:--:-- 19000
404 page not found
+ kill 87617
+ run_lp -broadcaster -httpIngest=false
+ pid=87632
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -httpIngest=false
*-------------*-------*
| HttpIngest  | false |
| Broadcaster | true  |
*-------------*-------*
I1219 15:02:21.884357   87632 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:21.885186   87632 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:21.885222   87632 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:21.885270   87632 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:21.885286   87632 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:21.885300   87632 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:21.885370   87632 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -X PUT http://localhost:9935/live/movie/0.ts
+ grep '404 page not found'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    19  100    19    0     0  28528      0 --:--:-- --:--:-- --:--:-- 19000
404 page not found
+ kill 87632
+ run_lp -broadcaster -httpAddr 0.0.0.0 -authWebhookUrl http://foo.com -httpIngest=false
+ pid=87648
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -httpAddr 0.0.0.0 -authWebhookUrl http://foo.com -httpIngest=false
*----------------*----------------*
| HttpAddr       | 0.0.0.0        |
| HttpIngest     | false          |
| Broadcaster    | true           |
| AuthWebhookURL | http://foo.com |
*----------------*----------------*
I1219 15:02:22.143530   87648 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:22.144319   87648 starter.go:565] ***Livepeer is in off-chain mode***
I1219 15:02:22.144366   87648 starter.go:1042] Using auth webhook URL http://foo.com
E1219 15:02:22.144390   87648 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:22.144450   87648 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:22.144467   87648 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:22.144492   87648 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:22.144582   87648 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -X PUT http://localhost:9935/live/movie/0.ts
+ grep '404 page not found'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    19  100    19    0     0  35647      0 --:--:-- --:--:-- --:--:-- 19000
404 page not found
+ kill 87648
+ run_lp -broadcaster -httpAddr 0.0.0.0 -httpIngest
+ pid=87663
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -httpAddr 0.0.0.0 -httpIngest
*-------------*---------*
| HttpAddr    | 0.0.0.0 |
| Broadcaster | true    |
*-------------*---------*
I1219 15:02:22.403416   87663 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:22.404210   87663 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:22.404247   87663 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:22.404294   87663 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:22.404310   87663 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:22.404324   87663 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:22.404377   87663 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -X PUT http://localhost:9935/live/movie/0.ts
+ grep -v '404 page not found'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0I1219 15:02:22.639388   87663 mediaserver.go:797] manifestID=movie clientIP=127.0.0.1 Got push request at url=http://localhost:9935/live/movie/0.ts ua=curl/7.81.0 addr=127.0.0.1 bytes=0 dur= resolution= slice-from= slice-to=
W1219 15:02:22.639630   87663 mediaserver.go:856] manifestID=movie clientIP=127.0.0.1 Couldn't detect input video stream codec
I1219 15:02:22.639681   87663 broadcast.go:768] manifestID=movie nonce=7477651996579005060 clientIP=127.0.0.1 No orchestrators specified; not transcoding
I1219 15:02:22.639705   87663 broadcast.go:768] manifestID=movie nonce=7477651996579005060 clientIP=127.0.0.1 No orchestrators specified; not transcoding
I1219 15:02:22.639715   87663 mediaserver.go:985] Missing duration; filling in a default of 2000ms
I1219 15:02:22.639737   87663 broadcast.go:1045] manifestID=movie nonce=7477651996579005060 seqNo=0 clientIP=127.0.0.1 No sessions available for segment
E1219 15:02:22.639749   87663 mediaserver.go:1048] manifestID=movie nonce=7477651996579005060 seqNo=0 clientIP=127.0.0.1 No sessions available name=0.ts url=http://localhost:9935/live/movie/0.ts
I1219 15:02:22.639769   87663 mediaserver.go:971] manifestID=movie nonce=7477651996579005060 seqNo=0 clientIP=127.0.0.1 Finished push request at url=http://localhost:9935/live/movie/0.ts ua=curl/7.81.0 addr=127.0.0.1:39878 bytes=0 dur= resolution= took=356.212µs
100    22  100    22    0     0  20992      0 --:--:-- --:--:-- --:--:-- 22000
No sessions available
+ kill 87663
+ run_lp -broadcaster -httpAddr 127.0.0.1
+ pid=87680
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -httpAddr 127.0.0.1
*-------------*-----------*
| HttpAddr    | 127.0.0.1 |
| Broadcaster | true      |
*-------------*-----------*
I1219 15:02:22.662817   87680 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:22.663607   87680 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:22.663645   87680 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:22.663692   87680 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:22.663707   87680 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:22.663722   87680 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:22.663794   87680 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -X PUT http://localhost:9935/live/movie/0.ts
+ grep -v '404 page not found'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0I1219 15:02:22.899934   87680 mediaserver.go:797] manifestID=movie clientIP=127.0.0.1 Got push request at url=http://localhost:9935/live/movie/0.ts ua=curl/7.81.0 addr=127.0.0.1 bytes=0 dur= resolution= slice-from= slice-to=
W1219 15:02:22.900185   87680 mediaserver.go:856] manifestID=movie clientIP=127.0.0.1 Couldn't detect input video stream codec
I1219 15:02:22.900233   87680 broadcast.go:768] manifestID=movie nonce=10073383348126920726 clientIP=127.0.0.1 No orchestrators specified; not transcoding
I1219 15:02:22.900263   87680 broadcast.go:768] manifestID=movie nonce=10073383348126920726 clientIP=127.0.0.1 No orchestrators specified; not transcoding
I1219 15:02:22.900277   87680 mediaserver.go:985] Missing duration; filling in a default of 2000ms
I1219 15:02:22.900308   87680 broadcast.go:1045] manifestID=movie nonce=10073383348126920726 seqNo=0 clientIP=127.0.0.1 No sessions available for segment
E1219 15:02:22.900335   87680 mediaserver.go:1048] manifestID=movie nonce=10073383348126920726 seqNo=0 clientIP=127.0.0.1 No sessions available name=0.ts url=http://localhost:9935/live/movie/0.ts
I1219 15:02:22.900359   87680 mediaserver.go:971] manifestID=movie nonce=10073383348126920726 seqNo=0 clientIP=127.0.0.1 Finished push request at url=http://localhost:9935/live/movie/0.ts ua=curl/7.81.0 addr=127.0.0.1:39880 bytes=0 dur= resolution= took=389.842µs
100    22  100    22    0     0  22022      0 --:--:-- --:--:-- --:--:-- 22000
No sessions available
+ kill 87680
+ run_lp -broadcaster -httpAddr localhost
+ pid=87696
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -httpAddr localhost
*-------------*-----------*
| HttpAddr    | localhost |
| Broadcaster | true      |
*-------------*-----------*
I1219 15:02:22.923132   87696 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:22.924083   87696 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:22.924131   87696 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:22.924203   87696 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:22.924226   87696 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:22.924244   87696 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:22.924319   87696 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -X PUT http://localhost:9935/live/movie/0.ts
+ grep -v '404 page not found'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0I1219 15:02:23.160166   87696 mediaserver.go:797] manifestID=movie clientIP=127.0.0.1 Got push request at url=http://localhost:9935/live/movie/0.ts ua=curl/7.81.0 addr=127.0.0.1 bytes=0 dur= resolution= slice-from= slice-to=
W1219 15:02:23.160402   87696 mediaserver.go:856] manifestID=movie clientIP=127.0.0.1 Couldn't detect input video stream codec
I1219 15:02:23.160454   87696 broadcast.go:768] manifestID=movie nonce=2670941761508980840 clientIP=127.0.0.1 No orchestrators specified; not transcoding
I1219 15:02:23.160483   87696 broadcast.go:768] manifestID=movie nonce=2670941761508980840 clientIP=127.0.0.1 No orchestrators specified; not transcoding
I1219 15:02:23.160493   87696 mediaserver.go:985] Missing duration; filling in a default of 2000ms
I1219 15:02:23.160517   87696 broadcast.go:1045] manifestID=movie nonce=2670941761508980840 seqNo=0 clientIP=127.0.0.1 No sessions available for segment
E1219 15:02:23.160529   87696 mediaserver.go:1048] manifestID=movie nonce=2670941761508980840 seqNo=0 clientIP=127.0.0.1 No sessions available name=0.ts url=http://localhost:9935/live/movie/0.ts
I1219 15:02:23.160553   87696 mediaserver.go:971] manifestID=movie nonce=2670941761508980840 seqNo=0 clientIP=127.0.0.1 Finished push request at url=http://localhost:9935/live/movie/0.ts ua=curl/7.81.0 addr=127.0.0.1:39892 bytes=0 dur= resolution= took=348.432µs
100    22  100    22    0     0  22750      0 --:--:-- --:--:-- --:--:-- 22000
No sessions available
+ kill 87696
+ run_lp -broadcaster -httpAddr 0.0.0.0 -authWebhookUrl http://foo.com
+ pid=87711
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -httpAddr 0.0.0.0 -authWebhookUrl http://foo.com
*----------------*----------------*
| HttpAddr       | 0.0.0.0        |
| Broadcaster    | true           |
| AuthWebhookURL | http://foo.com |
*----------------*----------------*
I1219 15:02:23.183231   87711 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:23.184069   87711 starter.go:565] ***Livepeer is in off-chain mode***
I1219 15:02:23.184113   87711 starter.go:1042] Using auth webhook URL http://foo.com
E1219 15:02:23.184131   87711 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:23.184186   87711 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:23.184203   87711 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:23.184220   87711 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:23.184301   87711 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -X PUT http://localhost:9935/live/movie/0.ts
+ grep -v '404 page not found'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0I1219 15:02:23.419778   87711 mediaserver.go:797] manifestID=movie clientIP=127.0.0.1 Got push request at url=http://localhost:9935/live/movie/0.ts ua=curl/7.81.0 addr=127.0.0.1 bytes=0 dur= resolution= slice-from= slice-to=
W1219 15:02:23.419990   87711 mediaserver.go:856] manifestID=movie clientIP=127.0.0.1 Couldn't detect input video stream codec
E1219 15:02:24.200713   87711 mediaserver.go:270] manifestID=movie nonce=6814034845274719556 clientIP=127.0.0.1 Authentication denied for streamID url=http://localhost:9935/live/movie/0.ts err="invalid character '<' looking for beginning of value"
E1219 15:02:24.200770   87711 mediaserver.go:749] Could not create stream ID: url=http://localhost:9935/live/movie/0.ts
100    70  100    70    0     0     89      0 --:--:-- --:--:-- --:--:--    89
Could not create stream ID: url=http://localhost:9935/live/movie/0.ts
+ kill 87711
+ run_lp -broadcaster
+ pid=87727
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster
*-------------*------*
| Broadcaster | true |
*-------------*------*
I1219 15:02:24.224639   87727 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:24.225561   87727 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:24.225601   87727 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:24.225651   87727 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:24.225667   87727 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:24.225683   87727 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:24.225720   87727 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -s --stderr - http://localhost:5935/getBroadcastConfig
+ grep P240p30fps16x9,P360p30fps16x9
{"MaxPrice":null,"TranscodingOptions":"P240p30fps16x9,P360p30fps16x9"}
+ kill 87727
+ run_lp -broadcaster -transcodingOptions P144p30fps16x9,P720p30fps16x9
+ pid=87742
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -transcodingOptions P144p30fps16x9,P720p30fps16x9
*--------------------*-------------------------------*
| Broadcaster        | true                          |
| TranscodingOptions | P144p30fps16x9,P720p30fps16x9 |
*--------------------*-------------------------------*
I1219 15:02:24.484047   87742 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:24.484893   87742 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:24.484933   87742 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:24.484994   87742 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:24.485012   87742 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:24.485026   87742 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:24.485114   87742 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -s --stderr - http://localhost:5935/getBroadcastConfig
+ grep P144p30fps16x9,P720p30fps16x9
{"MaxPrice":null,"TranscodingOptions":"P144p30fps16x9,P720p30fps16x9"}
+ kill 87742
+ cat
+ run_lp -broadcaster -transcodingOptions /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/profile.json
+ pid=87758
+ sleep 0.25
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -transcodingOptions /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/profile.json
*--------------------*-----------------------------------------------------------------------------------*
| Broadcaster        | true                                                                              |
| TranscodingOptions | /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/profile.json |
*--------------------*-----------------------------------------------------------------------------------*
I1219 15:02:24.743433   87758 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:24.744223   87758 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:24.744259   87758 starter.go:1099] No orchestrator specified; transcoding will not happen
I1219 15:02:24.744348   87758 starter.go:1264] ***Livepeer Running in Broadcaster Mode***
I1219 15:02:24.744365   87758 starter.go:1265] Video Ingest Endpoint - rtmp://127.0.0.1:1935
I1219 15:02:24.744380   87758 starter.go:1272] Livepeer Node version: undefined
I1219 15:02:24.744443   87758 webserver.go:20] CLI server listening on 127.0.0.1:5935
+ curl -s --stderr - http://localhost:5935/getBroadcastConfig
+ grep abc,def
{"MaxPrice":null,"TranscodingOptions":"abc,def"}
+ kill 87758
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -transcodingOptions notarealfile
+ grep 'No transcoding profiles found'
E1219 15:02:25.004764   87773 starter.go:1588] Error creating Livepeer server: err="No transcoding profiles found"
+ echo 'not json'
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -transcodingOptions /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/invalid.json
+ grep 'invalid character'
E1219 15:02:25.030463   87788 starter.go:1588] Error creating Livepeer server: err="invalid character 'o' in literal null (expecting 'u')"
+ echo '[{"width":"1","height":"2"}]'
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -transcodingOptions /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/schema.json
+ grep 'cannot unmarshal string into Go struct field JsonProfile.width of type int'
E1219 15:02:25.055630   87801 starter.go:1588] Error creating Livepeer server: err="json: cannot unmarshal string into Go struct field JsonProfile.width of type int"
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -transcodingOptions invalid
+ grep -v 'Local verification enabled'
*--------------------*---------*
| Broadcaster        | true    |
| TranscodingOptions | invalid |
*--------------------*---------*
I1219 15:02:25.080063   87814 starter.go:407] ***Livepeer is running on the offchain network***
I1219 15:02:25.081002   87814 starter.go:565] ***Livepeer is in off-chain mode***
E1219 15:02:25.081043   87814 starter.go:1099] No orchestrator specified; transcoding will not happen
E1219 15:02:25.081095   87814 starter.go:1588] Error creating Livepeer server: err="No transcoding profiles found"
+ /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057/livepeer -broadcaster -transcodingOptions invalid -localVerify=true
+ grep 'Local verification enabled'
I1219 15:02:25.106688   87828 starter.go:1140] Local verification enabled
+ rm -rf /home/ricks/development/personal/go-livepeer/tmp/livepeer-test-17057
+ printf '\n\nAll Tests Passed\n\n'


All Tests Passed

leszko
leszko reviewed Dec 20, 2023
View reviewed changes
monitor/census.go Outdated Show resolved Hide resolved
@rickstaa rickstaa force-pushed the add_reward_service_error_monitor_parameter branch 2 times, most recently from 364cc81 to a4a4889 Compare December 20, 2023 12:54
@rickstaa
feat: add round in reward call error log statement
8acadbc 
This commit improves the error that is thrown when the reward call fails
by including the round number.
@rickstaa rickstaa force-pushed the add_reward_service_error_monitor_parameter branch from a4a4889 to 8acadbc Compare December 20, 2023 13:07
@rickstaa
Copy link
Contributor Author

@leszko I applied your suggestions and think this pull request can now be merged. The metrics are now displaying correctly, as evidenced by the screenshot linked below:

image

Also, I've decided to exclude the transaction hash from the display. This information is redundantly included in the error messages when a transaction fails, so it seemed unnecessary to duplicate it. You can see this in the following screenshot:

image

Please review the changes whenever you get a chance. The implementation aligns well with our objectives and is ready for integration.

leszko
leszko approved these changes Dec 20, 2023
View reviewed changes
Copy link
Contributor

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rickstaa fr all the changes.

@leszko leszko merged commit d877233 into livepeer:master Dec 20, 2023
14 of 15 checks passed
@rickstaa rickstaa deleted the add_reward_service_error_monitor_parameter branch December 20, 2023 14:05
@rickstaa
Copy link
Contributor Author

@leszko, thanks for the review and merging this in the main branch.

@thomshutt thomshutt mentioned this pull request Jan 29, 2024
Merged
eliteprox pushed a commit to eliteprox/go-livepeer that referenced this pull request Feb 21, 2024
@rickstaa @eliteprox
feat(monitor): add reward service error metric (livepeer#2905)
7af7020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leszko leszko leszko approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rickstaa @eliteprox @leszko